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PREFACE 


This  document  was  prepared  by  the  Institute  for  Defense  Analyses  (IDA)  for  the 
Director,  Operational  Test  and  Evaluation,  in  partial  response  to  the  task  Assessment  of 
Electronic  Warfare  Systems.  The  objective  of  this  effort  was  to  summarize  the  concept 
and  results  of  the  use  of  the  Airborne  Instrumentation  System  (AIS)  for  electronic  combat 
test  and  evaluation.  The  AIS  was  funded  by  the  Resource  Enhancement  Program  (REP) 
and  sponsored  by  DOT&E.  This  paper  will  be  presented  to  the  International  Test  and 
Evaluation  Association  (ITEA)  workshop  “Modeling  and  Simulation  -  1996  and  Beyond 
...  Are  We  Progressing?”  on  9  through  12  December  1996. 

The  IDA  Review  Committee  consisted  of  Mr.  Thomas  P.  Christie,  Director  of  the 
Operational  Evaluation  Division,  and  Dr.  Alfred  E.  Victor,  project  leader  for  the  task, 
who  also  provided  technical  oversight. 
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AIRBORNE  INSTRUMENTATION  SYSTEM  (AIS)  FOR 
ELECTRONIC  COMBAT  TEST  AND  EVALUATION: 
CONCEPT  AND  VALIDATION 

A.  THE  PROBLEM  WITH  GARLIC 

One  of  the  fundamental  difficulties  in  the  test  and  evaluation  (T&E)  of  an 
electronic  combat  (EC)  system  is  that  it  is  a  “soft-kill”  device.  EC  is  very  much  like 
vampires  and  garlic;  if  you  wear  a  garland  of  garlic  around  your  neck  and  are  not 
attacked  by  a  vampire,  you  can  claim  that  it  was  the  garlic  that  prevented  it.  However, 
you  will  have  a  difficult  time  convincing  the  skeptics.  Similarly,  at  the  end  of  even  the 
most  authentic  test  of  an  EC  component,  there  is  often  no  hard  evidence  that  the 
component  functioned  properly.’  At  most,  there  may  be  some  negative  evidence  that 
something  was  not  right;  for  example,  a  missile  might  shoot  down  a  drone  equipped  with 
the  system.  But  proving  that  an  EC  component  failure  caused  the  drone  to  be  shot  down 
is  difficult.  Therefore,  to  test  an  EC  component  adequately,  a  substantial  amount  of 
information  must  be  gathered  about  the  test  environment  and  the  response  of  the  system 
under  test. 

One  of  the  most  significant  problems  in  addressing  this  requirement  is  the  lack  of 
adequately  instrumented  aircraft.  A  limited  number  of  instrumented  aircraft  dedicated  to 
testing  do  exist,  but  the  instrumentation  is  expensive  and  often  intrusive.  There  also  is  a 
strong  desire  to  use  combat  representative  aircraft  during  some  portions  of  testing, 
particularly  the  latter  phases  of  operational  test  and  evaluation  (OT&E).  Unfortunately, 
the  instrumentation  of  these  aircraft  typically  consists  of  videotape  recorders  and 
kneeboard  cards.  The  data  are  incomplete,  time  consuming  to  reduce  and  analyze,  and 
make  unambiguous  correlation  of  environmental  stimuli  and  system  response  impossible. 

Complicating  the  instrumentation  problem  further  is  the  variety  of  data  collection, 
reduction,  and  analysis  procedures,  often  created  from  scratch,  to  suit  the  peculiarities  of 
each  test.  Even  within  the  same  acquisition  program,  data  are  rarely  collected,  reduced 


’  Or  improperly,  for  that  matter.  No  EC  system  is  perfect,  and  a  kill  by  a  threat  system  does  not  prove 
that  the  EC  system  failed.  (A  test  in  which  the  platform  is  killed  every  time  is  pretty  convincing, 
however.) 


and  analyzed  consistently  in  all  phases  of  the  test.  This  leads  to  disagreement  over  the 
system’s  performance  in  different  test  program  phases. 

Many  of  the  existing  data  collection,  reduction,  and  analysis  processes  require  a 
week  or  more  before  the  data  are  available  for  initial  review.  Often  several  more  flights 
have  occurred,  which  means  that  a  flawed  portion  of  the  test  may  continue  to  invalidate 
several  more  missions  before  it  is  detected.  In  several  recent  EC  operational  tests,  serious 
flaws  in  the  instrumentation  or  data  reduction  have  caused  the  loss  of  up  to  half  of  the 
data  collected. 

B.  THE  NEED  FOR  A  SOLUTION 

To  address  the  EC  test  data  collection  problem,  a  multi-Service  (“purple”), 
multi-platform,  multi-system  package  for  aircraft  instrumentation  that  is  non-invasive, 
flexible  and  easy  to  use,  and  able  to  provide  quick-look  results  is  needed.  Ideally,  it 
should  provide  non-invasive  and  selective  aircraft  bus  monitoring  and  require  minimal  or 
no  modification  of  all  operational  aircraft  of  interest.  It  should  be  able  to  operate 
independent  of  the  test  range  infrastructure  and  yet  provide  accurate  position  information. 
It  should  be  easy  to  configure  and  verify  its  correct  operation.  Its  use  should  have 
minimal  impact  on  pre-flight  and  post-flight  procedures.  The  supporting  and  analysis 
software  should  be  usable  for  mission  visualization,  simple  automated  processing,  and  in 
a  more  detailed  mode  allowing  alternate  analyses  of  the  data. 

C.  A  SOLUTION  -  THE  AIRBORNE  INSTRUMENTATION  SYSTEM  (AIS) 

After  witnessing  several  inadequate  tests  in  the  EC  OT&E  testing  area,  the 
Director  of  Operational  Test  and  Evaluation  (DOT&E)  in  OSD  decided  to  develop  a 
system  of  hardware  and  software  to  address  these  needs.  The  system  is  the  Airborne 
Instrumentation  System  (AIS).  The  initial  focus  for  this  “purple”  pod  was  for  the  T&E 
instrumentation  needs  of  any  EC  system;  in  the  future,  it  could  be  used  for  other  avionics 
systems.  In  its  present  configuration,  it  is  composed  of  two  pieces  of  hardware  to  cope 
with  different  aircraft  installations,  a  flightline  computer,  and  three  software  components. 
The  external  Test  Instrumentation  Pod  (TIP)  was  designed  and  built  by  Metric  Systems 
Corporation  and  is  based  on  a  P4B  Air  Combat  Maneuvering  Information  (ACMI)  pod. 
Five  were  built  as  part  of  the  AIS  program.  The  internal  Alternate  Data  Acquisition 
System  (ADAS)  is  based  on  a  Canadian  design  (the  Record  All  Small  Data  Acquisition 
System  (RASDAS));  only  one  of  these  was  assembled  by  W.J.  Associates.  The  software 
was  developed  by  the  96th  CCSG/SCWA,  Eglin  Air  Force  Base  (the  “Math  Lab”). 


Sponsorship  for  the  program  is  provided  by  DOT&E,  supported  by  IDA  and  life  cycle 
support  is  being  provided  by  the  96^*^  CCSG/SCW  at  Eglin  Air  Force  Base. 

Independently  of  the  AIS  program,  COMPTEK  Systems  developed  the  Digital 
Bus  Recorder  System  (DBRS),  another  internal  recorder.  The  data  from  this  system  is 
compatible  with  the  software  developed  by  AIS. 

D.  GENESIS  OF  AIS 

The  idea  for  the  AIS  originated  with  a  software/hardware  combination  developed 
by  Georgia  Tech  Research  Institute  (GTRI).  Based  on  a  modified  USAF  general 
instrumentation  pod,  the  Digital  Flight  Recorder  (DFR)  includes  a  Coarse/Acquisition 
(C/A)  code  Global  Positioning  System  (GPS)  receiver  and  antenna,  processor,  static 
RAM,  and  an  interface  with  the  aircraft  MIL-STD-1553  bus.  The  DFR  could  be  rapidly 
loaded/unloaded  from  an  aircraft  and  was  easily  configurable  from  a  portable  PC.  The 
data  could  be  downloaded  to  the  same  PC  at  the  end  of  the  flight  and  imported  into 
GTRTs  Automated  Data  Reduction  System  (ADRS)  software,  together  with  threat 
simulator  on/off  data,  and  analyzed  the  same  day  the  mission  was  flown. 

There  are  several  significant  problems  with  the  hardware  and  software, 
particularly  for  the  joint  Service  use  requirement.  The  DFR  is  16  inches  in  diameter  and 
weighs  approximately  450  pounds,  significantly  impacting  aircraft  flying  qualities.  The 
early  generation  GPS  receiver  is  prone  to  dropouts  in  reception  of  the  GPS  signals.  The 
ADRS  software  is  limited  to  analyzing  tightly  constrained  missions  because  of  a  very 
limited  representation  of  emitter  activity.  The  stimulus-response  algorithm  does  not 
include  emitter  radio  frequency  (RF)  parameter  correlation  or  whether  the  aircraft  was 
within  the  emitter’s  beam.  The  software  does  not  allow  for  flexible  data  import  or 
export.  The  software  is  undergoing  a  significant  upgrade  that  may  address  these  issues. 

E.  CONCEPT  OF  USE  FOR  AIS 

A  more  general  concept,  based  on  the  GTRI  approach,  was  developed  for  AIS,  as 
illustrated  in  Figures  1  and  2.  Figure  1  shows  the  use  of  AIS  at  any  test  range  with 
emitters  illuminating  the  aircraft.  The  AIS  collects  MIL-STD-1553  bus  data  from  an 
on-board  radar  warning  receiver  (RWR)  (or  other  avionics  system),  GPS  position  data 
and  inertial  position  and  attitude  data  from  either  the  aircraft  or  an  AIS  sub-system. 
Likewise,  for  data  correlation,  the  emitters  illuminating  the  aircraft  collect  data  on  their 
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Figure  1.  TIP  Use  During  T&E 
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Figure  2.  Data  Path  from  TIP  to  Analyzed  Data 


modes  and  pointing  angles  tagged  with  GPS  time.  If  data  correlation  is  less  important, 
emitter  on  and  off  times  may  be  logged  by  hand  and  normal  operating  ranges  assumed  for 
emitter  RF  parameters.  After  the  aircraft  lands,  the  data  are  downloaded  with  a 
removable  medium.  These  data  are  translated  into  engineering  units,  combined  with 
emitter  truth  data,  and  analyzed  on  a  PC.  Depending  on  how  fast  the  emitter  data  can  be 
gathered,  analysis  of  the  data  can  begin  as  early  as  a  few  hours  after  the  aircraft  lands. 
The  system  is  modular  so  that  components  can  be  replaced  or  omitted,  such  as  using  a 
different  data  recorder  or  using  test  range  time,  space,  and  position  information  (TSPI) 
data  versus  data  collected  on-board  or  omitting  aircraft  attitude  data,  without  causing  the 
process  to  collapse. 

F.  DESCRIPTION  OF  AIS  COMPONENTS 
1.  TIP  Hardware 

The  TIP  is  based  on  a  modified  ACMI  P4B  instrumentation  pod  (it  has  similar 
form,  fit  and  moments  of  inertia  as  an  AIM-9)  that  has  had  several  components  removed 
(see  Figure  3).  In  their  place  an  all-in-view  GPS  receiver  using  the  existing  antenna,  a 
high-reliability  fiber  optic  gyroscope  (FOG)  inertial  measurement  unit  (IMU),  and  a 
navigational  processor  unit  (NPU)  for  producing  smoothed  position  data  have  been 
added.  An  Intel-based  Digital  Interface  and  Processor  Unit,  using  information  stored  in  a 
configuration  file,  is  used  for  filtering  and  managing  the  TSPI  data  and  two 
dual-redundant  1553  buses.  All  data  are  stored  on  two  removable  Personal  Computer 
Memory  Card  Industry  Association  (PCMCIA)  cards  (total  current  capacity  80  MB) 
stored  in  the  rear  of  the  pod.  These  allow  for  rapid  data  downloading,  high  capacity,  and 
simple  declassification  of  the  TIP.  The  rear  of  the  pod  has  been  modified  to  incorporate 
an  indicator  light  for  pod  built-in  test  (BIT)  results,  a  connector  for  uploading 
configuration  files,  and  a  PCMCIA  card  drive  and  cover.  The  ACMI  transponder  was 
retained,  allowing  the  pod  to  function  as  an  ACMI  pod.  Unlike  current  ACMI  pods,  it 
will  have  recording  capability  and  GPS  position  data  in  case  data  drop-outs  corrupt  the 
telemetry  data.  When  functioning  as  an  ACMI  pod,  the  TIP  will  be  able  to  act  as  a  1553 
remote  terminal  as  well  as  a  bus  monitor.  The  transponder  can  be  disabled  for 
compatibility  or  security  reasons  via  the  configuration  file. 
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Figure  3.  AIS  TIP  Components 


The  TIP  mounts  to  a  standard  LAU-7  missile  launcher  and  uses  standard  cabling. 
If  EW  bus  data  are  available  at  the  weapons  station,  no  modification  to  the  aircraft  is 
required.  Otherwise,  a  wiring  modification  must  be  made  or  a  different  recording  system 
form  factor  must  be  used  (see  below  for  the  approach  taken  for  the  F/A-18).  On  the 
F-14D,  this  modification  requires  a  single  cable  and  approximately  two  hours  to  install. 
It  can  be  left  in  until  that  weapons  station  is  needed  for  a  weapons  delivery,  then 
removed. 

The  GPS  receiver  is  a  12-channel  C/A  code  NovAtel  model  3151  receiver. 
Although  the  signals  from  only  four  satellites  are  needed  to  produce  a  TSPI  solution,  the 
additional  channels  are  used  to  track  extra  satellites  so  that  the  best  four  satellites  can  be 
used  to  calculate  a  solution  under  all  conditions.  The  additional  satellites  are  also  useful 
so  that  if  a  satellite  being  used  for  the  current  solution  drops  out  of  view,  another  will  be 
available  immediately  to  incorporate  into  the  solution.  Finally,  if  as  many  as  eight  or 
more  satellites  are  available,  they  can  be  used  to  produce  a  second  solution  that  can  be 
used  to  provide  quality  estimates  on  the  primary  solution. 
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The  GPS  receiver  and  the  FOG  IMU  are  loosely  coupled  via  the  NPU  to  produce 
an  aided  absolute  solution,  replacing  the  original  ACMI  pod  inertial  sensor  assembly. 
The  FOG  is  a  very  reliable  system  since  it  has  no  moving  parts  and  is  likely  to  become 
the  standard  inertial  data  source  for  GPS  aiding.  GPS  receivers  and  IMUs  have 
complementary  strengths.  GPS  receivers  produce  very  accurate  position  data,  but  are 
prone  to  high  frequency  and  large  amplitude  errors  and  have  a  low  data  rate,  whereas  an 
IMU  is  subject  only  to  a  very  low  frequency  drift,  allows  coasting  during  loss  of  GPS 
receiver  lock,  and  can  produce  data  at  a  higher  rate.  Together  they  allow  the  calculation 
of  a  Method  3  differential  aided  solution  post-mission  using  a  reference  receiver. 

2.  ADAS  Hardware 

The  Alternate  Data  Acquisition  System  (ADAS)  system  was  designed  as  an 
interim  instrumentation  system  to  fill  a  need  for  an  internal  bus  recorder  on  the  F/A-18. 
The  ADAS  is  also  a  potential  solution  for  bus  instrumentation  on  other  aircraft  where 
accessing  the  desired  1553  data  bus  at  a  weapons  station  is  not  possible,  or  where 
carrying  an  AIM-9  form  pod  is  not  possible.  In  normal  operation  it  is  complemented  by  a 
TSPI  data  source  (external  GPS  pod  or  test  range  tracking  systems).  It  consists  of  a 
Merlin  data  encoder  for  converting  1553  data  into  RS-170  video  format  for  recording  on 
a  videotape  recorder  (VTR),  an  IRIG-B  time  code  generator  and  a  TEAC  Hi-8  millimeter 
VTR  (providing  two  hours  of  recording  time).  These  components  (plus  a  power 
conditioner)  are  packaged  for  mounting  inside  the  gun  bay  access  door  without  displacing 
the  gun,  ammo  tray  or  any  other  systems.  The  gun  bay  location  was  chosen  because 
several  1553  buses  (weapons,  avionics,  and  EW)  have  connections  there. 

Recording  begins  at  power  up.  The  time  synch  for  the  IRIG  time  code  generator 
is  provided  by  a  temporarily  connected  hand-held  GPS  receiver.  Upon  mission 
completion,  the  Hi-8  millimeter  tape  is  removed  and  taken  to  a  computer  equipped  with 
an  Advanced  Bus  Interface  (ABI)  Protocol  Analysis  Simulation  System  (PASS)  PC  card 
and  PASS  1000  software  (manufactured  by  SB S.  Engineering).  A  Merlin  decoder  and 
VTR  are  used  to  replay  the  data  into  the  computer  where  it  is  decoded  and  the  desired 
1553  messages  are  selected.  This  ground  station  can  be  used  also  as  a  bus  monitor 
directly  connected  to  an  aircraft  1553  bus  (for  diagnostic  use  on  the  ground).  The  data 
can  then  be  translated  by  Common  Airborne  Processing  System  (CAPS)  and  analyzed  by 
Mission  Analysis  and  Reporting  System  (MARS)  or  other  analysis  packages. 


3.  COMPTEK  DBRS  Hardware 


Since  ADAS  has  some  fundamental  limitations  (no  data  filtering  capability  on 
record,  no  remote  turn  on  capability,  limited  recording  time),  other  internal  systems  were 
considered  for  future  development  by  the  AIS  program.  A  functional  design  similar  to 
the  TIP  pod  including  a  processor  for  filtering,  managing  data,  and  PCMCIA  card  storage 
was  considered.  Before  funding  could  be  located  for  this,  COMPTEK  Systems  designed 
the  Digital  Bus  Recorder  System  (DBRS)  exactly  fitting  these  specifications. 
Substantially  smaller  than  the  ADAS,  it  can  fit  also  in  the  gun  bay  of  an  F/A-18  and  uses 
an  external  time  synchronization.  Rather  than  an  IRIG-B  time  card,  the  DBRS  uses  the 
clock  on  the  Motorola  68030-based  logic  board,  which  allows  a  drift  of  about  one  second 
per  day.  The  current  configuration  consists  of  a  single  170  MB  PCMCIA  card.  Two  of 
these  systems  were  purchased  by  AIRTEVRON  9  at  China  Lake,  one  of  the  Navy’s  test 
and  evaluation  squadrons.  Because  of  this,  the  DBRS  has  been  included  in  the  list  of 
recorders  compatible  with  the  AIS  software  components. 

4.  AIS  Software/Support  Systems 

Several  pieces  of  software  have  been  developed  under  the  AIS  program  to  assist 
in  the  uploading  and  downloading  of  data,  translation  of  the  data  into  engineering  units 
and  standard  formats,  analysis,  and  reporting.  Metric  developed  the  upload  software  for 
the  modification  of  the  TIP  configuration  file  that  controls  what  data  are  recorded  and  at 
what  rate.  1 553  data  can  be  filtered  down  to  the  remote  terminal  address,  subaddress,  and 
word  level.  M  out  of  N  sampling  is  allowed  to  reduce  the  data  rate  from  some  systems 
(inertial  data  in  particular). 

Three  pieces  of  software  have  been  written  or  significantly  modified  by  the  Eglin 
Math  Lab  to  translate,  correlate,  and  reduce  the  data  from  the  recorders;  the  Common 
Airborne  Processing  System  (CAPS),  the  Mission  Analysis  and  Reporting  System 
(MARS),  and  the  GPS  analyst  software. 

CAPS  is  a  general  purpose  data  translation  program  for  converting  binary  data  to 
formatted  data  with  engineering  units.  It  uses  a  modular  architecture  to  allow  additional 
bus  data  formats  to  be  interpreted  and  additional  output  formats  to  be  specified. 
Currently,  CAPS  interprets  IRIG  PCM  Class  I/II,  MIL-STD-1553  message  data,  and 
several  GPS  receiver  formats.  A  system-specific  data  dictionary  specifies  the  translation 
to  be  performed  for  all  recorded  messages.  It  identifies  the  command  word,  word  and  bit 
ranges,  the  binary  data  type  (e.g.,  two’s  complement),  and  the  proper  conversion  factor  to 
be  applied  to  translate  each  item.  The  information  to  produce  this  data  dictionary  should 


be  contained  in  a  system’s  Interface  Control  Document  (ICD).  Once  the  data  are 
imported  and  translated  in  CAPS,  subsets  can  be  selected  and  exported  using  Output 
Product  Description  files  (OPD)  to  specify  the  variables,  order,  format  and  delimiters  best 
suited  for  the  analysis.  The  dictionaries  and  OPDs  can  be  created  by  the  user.  In  some 
cases,  where  the  relationships  between  the  variables  are  complex,  a  dynamic  link  library 
(DLL)  file  must  be  used  to  process  some  of  the  data  during  the  export  step. 

MARS  is  an  RWR  and  self-protection  electronic  countermeasures  (ECM)  system 
analysis  program  that  was  first  developed  for  the  F-15  Tactical  Electronic  Warfare 
System  (TEWS)  program  to  analyze  the  RWR  and  jammer  test  results.  The  jammer 
analysis  capability  was  developed  first,  with  the  additions  to  the  RWR  capability  added 
more  recently.  Much  of  the  future  capability  to  be  added  will  be  in  the  RWR  area. 

MARS  requires  data  to  be  formatted  in  predefined  database  tables  containing 
information  on  threat  system  actions,  aircraft  position  and  attitude,  and  system  response 
for  each  mission.  Depending  on  the  analyses  to  be  performed,  several  supporting  tables 
may  also  be  required  that  identify  emitter  locations,  emitter  RF  characteristics,  and 
translation  tables  for  system  response  (e.g.,  RWR  symbol  data  table  and  threat  pointer 
table).  Standard  database  operations  such  as  record  exclusion,  sorting,  searching, 
exporting  subsets  of  data,  and  calculations  on  various  fields  can  be  performed.  MARS 
includes  a  variety  of  jammer  and  RWR  analysis  methods,  including: 

•  RWR  display  simulation  with  user  editable  characters  and  enhancements 

•  Polar  and  strip  chart  plots  of  any  numeric  fields 

•  SAM/AAA  shot  analysis  (including  AAA  burst  analysis) 

•  Computation  of  reduction  in  hits  and  survivability  measures 

•  Correlation  of  RWR  response  to  threat  signal  characteristics 

'  ♦  Response  time/symbol  ageout  time/detection  range  analysis 

•  RWR  Direction  Finding  (DF)  error  analysis  (numerical  and  graphical) 

•  Non-parametric  statistics  on  any  field 

•  Tracking  error  statistics. 

To  make  the  various  RWR  analyses  possible,  MARS  has  a  flexible  correlation 
capability.  PRJ,  frequency  and  pulse  width  as  measured  by  the  RWR  can  be  correlated  in 
any  logical  combination  with  a  database  of  known  emitter  characteristics.  RWR  azimuth, 
threat  tracking  error,  and  threat  activity  (on/off)  data  can  be  included  in  the  correlation 
also. 
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As  a  QA  method  and  for  immediate  feedback  on  RWR  performance,  a  simulated 
display  with  the  displayed  symbols  and  enhancements  is  available.  The  specific  displays 
of  several  Air  Force  and  Navy  RWRs  have  been  created  so  that  the  appropriate 
symbology  and  range  rings  can  be  seen. 

The  final  software  component,  GPS  analyst,  integrates  GPS  and  IMU  data  for  the 
aircraft  position  with  data  from  a  ground-based  reference  receiver  to  produce  Method  3 
aided  differential  GPS  solutions  for  aircraft  position  when  more  accurate  TSPI  data  are 
needed.  The  output  of  this  program  can  be  used  by  MARS  for  EC  system  analyses. 

G.  RESULTS  OF  AIS  TESTING  TO  DATE 

The  AIS  and  the  supporting  software  have  been  in  almost  continuous  use  for  the 
last  two  years.  As  is  common  in  system  development,  there  have  been  some  problems, 
some  of  which  were  outside  the  control  of  the  AIS.  However,  they  do  reveal  the  kinds  of 
issues  that  have  to  be  addressed.  The  robustness  of  the  AIS  concept  has  been 
demonstrated  also.  Portions  of  the  AIS  have  been  used  on  eight  different  aircraft  in  two 
years.2  The  following  is  a  summary  of  the  results  from  these  tests. 

•  The  TIP/CAPS/MARS  combination  produces  results  identical  to  those 
captured  by  an  All-Bus  Recorder  and  the  standard  Patuxent  River  data 
reduction  process  on  an  instrumented  F-14D. 

•  A  C/A  code  GPS  receiver  with  Inertial  Navigational  System  (INS)  aiding  and 
differential  correction  can  provide  accuracies  better  than  30  feet  under  high 
dynamic  conditions. 

•  GPS  time  and  UTC  differ  by  a  number  of  leap  seconds  that  changes  roughly 
yearly.  It  is  currently  1 1  seconds. 

•  All  test  data  should  be  examined  immediately  after  every  mission.  Even 
though  AIS  gives  this  capability,  several  costly  mistakes  were  made  when 
this  capability  was  not  taken  advantage  of. 

•  Even  reference  systems  break  and  have  errors  that  must  be  accounted  for  in 
comparisons.  Several  AIS  calibration  tests  had  to  be  repeated  because  the 
reference  systems  failed. 

•  The  data  recording,  reduction,  and  analysis  chain  must  be  validated  from 
start  to  finish  prior  to  the  start  of  every  test  phase.  The  entire  chain  with  the 


2 


Aircraft  (and  EC  systems)  tested  to  date  include  F-16C  Block  40  (ALR-56M),  F-14D  (ALR-67E  (V)2 
and  ASPJ),  US  and  Canadian  F/A-18  (ALR-67E  (V)2  and  ALR-67  (V)3),  UH-IN  (APR-39 A),  and 
German  Air  Force  F-4,  Tornado,  and  MiG-29. 


AIS  was  not  fully  exercised  until  late  in  the  testing,  which  revealed  a  number 
of  previously  undetected  problems. 

•  MARS  needs  further  development  to  be  able  to  analyze  data  adequately  in  a 
complex  environment  or  for  systems  that  make  coarse  measurements  of  the 
RF  environment.  In  addition,  the  MARS  software  still  has  a  steep  learning 
curve  and  does  not  provide  sufficient  quicklook  capability.  These  issues  are 
being  addressed  in  upgrades  this  fiscal  year. 

•  CAPS  has  proven  itself  to  be  a  very  flexible  data  translation  tool.  For 
example,  ALR-67  (V)3  data  from  two  different  recorders  were  collected  over 
two  weeks  at  Patuxent  River’s  Air  Combat  Environment  Test  and  Evaluation 
Facility  (ACETEF)  chamber.  The  data  from  both  recorders  were  translated 
and  analysis  begun  two  days  after  the  first  data  were  collected. 

H.  LESSONS  LEARNED  DURING  DEVELOPMENT 

Some  important  issues  have  surfaced  while  developing  the  hardware  and  software 
components  for  the  AIS.  These  issues  are  generic  to  this  sort  of  instrumentation  system 
and  not  specific  to  AIS.  They  will  have  to  be  faced  for  each  new  aircraft  and/or  new 
system  to  be  instrumented  with  the  AIS  or  any  similar  system.  Although  they  appear  to 
be  simple  lessons,  experience  shows  that  they  bear  repeating. 

•  A  flexible  system  architecture  that  accommodates  multiple  hardware  and 
software  configurations  is  essential.  The  second  aircraft  examined  by  the 
AIS  program,  the  F/A-18,  cannot  be  conveniently  instrumented  with  the 
external  TIP  since  no  solution  for  bringing  electronic  warfare  (EW)  bus  data 
to  a  weapons  station  has  been  found. 

•  1553  bus  data  were  not  intended  to  be  translated  by  normal  humans.  The 
ICDs  to  translate  these  data  can  be  difficult  to  obtain  and  are  difficult  to 
interpret  unambiguously.  For  each  new  system  to  be  tested,  or  each  major 
OFP  upgrade,  several  man-weeks  must  be  spent  up  front  developing  a  CAPS 
data  dictionary  from  the  ICD  to  translate  the  data  into  engineering  units. 
This  is  still  substantially  less  time  than  developing  a  data  analysis  package 
from  scratch. 

•  Simple  operational  problems  can  defeat  the  best  technology.  The  data  from 
several  sorties  were  lost  because  of  systems  not  being  turned  on  or  being 
incorrectly  connected. 

•  Any  universal  instrumentation  and  analysis  system  must  prove  its  worth  by 
being  easier  and  less  expensive  to  use  than  existing  solutions.  Just  being  a 
“standard”  is  not  enough. 

•  To  produce  adequately  stable  TSPI  results,  a  GPS  receiver  must  be  aided  by 
an  IMU  for  high  dynamic  maneuvers.  However,  integrating  a  GPS  and  IMU 


system  is  still  an  art  form,  and  requires  many  revisions.  Several  of  the  early 
TIP  tests  revealed  significant  problems  in  this  area. 

•  Although  CAPS  and  MARS  are  being  designed  to  reduce  and  analyze  data 
from  a  wide  variety  of  systems,  experience  to  date  has  shown  that  every 
system  has  some  peculiarity  that  requires  additions  or  modifications  to  the 
CAPS/MARS  data  reduction/analysis  process.  To  date,  once  these 
peculiarities  are  identified,  solutions  have  been  available.  Any  universal 
system  will  have  to  deal  with  such  problems. 

I.  FUTURE  PLANS  AND  POSSIBILITIES  FOR  AIS 

There  are  plans  to  substantially  improve  CAPS,  MARS  and  GPS  analyst  this 
fiscal  year.  The  upgrade  of  CAPS  to  version  2.0  is  fully  funded  and  will  ship  May  1997. 
It  will  include  the  capability  to  handle  more  binary  data  types,  the  ability  to  work  in  a 
variety  of  data  formats  rather  than  importing  into  a  standard  format  (thus  saving  time), 
and  more  robust  capabilities  to  examine  translated  data  for  verification  that  the  translation 
has  gone  correctly. 

Several  phases  of  upgrades  to  MARS  are  being  considered.  As  currently 
envisioned,  the  first  phase  would  include  a  number  of  usability  and  functionality 
improvements  to  ease  analysis  of  a  broader  range  of  systems,  with  the  primary  addition 
being  a  “smart”  simulated  RWR  display  with  truth  data  displayed  for  reference.  The 
second  phase  would  be  the  addition  of  a  robust  scenario  map  function  to  ease  qualitative 
analysis  of  the  data  and  integrate  data  from  the  many  tables  in  MARS  into  a  single 
display.  The  final  phase  would  be  an  overhaul  of  the  correlation  function  and  support  file 
structure  to  allow  even  more  robust  matching  of  RWR-measured  signal  characteristics 
with  test  range  truth  data.  The  first  two  phases  of  this  plan  may  be  funded  shortly,  the 
final  phase  is  unfunded.  Also  currently  unfunded  is  the  completion  of  the  capability  to 
include  the  emitter  activity  and  position  information  for  up  to  10  airborne  threats. 

The  GPS  analyst  upgrades  are  still  being  determined.  They  will  focus  on 
broadening  its  capabilities  to  incorporate  TSPI  data  from  a  variety  of  sources  rather  than 
just  GPS  and  integrating  them  with  the  Advanced  Test  Data  Optimal  Processor 
(ATDOP).  Future  development  of  the  AIS  will  depend  on  the  requirements  of  the 
development  and  test  communities. 

1.  Possible  Future  Uses  of  AIS 

Although  the  focus  of  the  AIS  development  has  been  on  instrumenting  and 
analyzing  EC  systems  under  test,  the  availability  of  an  easy  to  use  bus  recorder. 


stand-alone  TSPI  source,  and  independent  data  reduction  and  analysis  capability  suggest 
several  other  possible  uses. 

•  Instrumenting  the  fire  control  radar  (FCR)  messages  on  aircraft  that  are 
acting  as  interceptors  during  a  test.  The  EC  system  response  could  be 
accurately  correlated  to  the  FCR  activity  using  CAPS/MARS. 3 

•  The  AIS  could  act  as  a  poor-man’s  intelligence  gathering  device,  allowing 
quick  looks  at  RF  parameters  gathered  from  targets  of  opportunity. 

•  The  ACMI  message  format  contains  a  number  of  spare  words.  By 
monitoring  both  the  EW  bus  and  the  weapons  bus  and  using  the  ACMI 
capability  of  the  TIP,  information  about  the  EC  system  under  test  could  be 
included  in  the  standard  ACMI  messages. 

J.  CONCLUSION 

The  AIS  is  becoming  the  multi-Service,  multi-aircraft,  multi-system 
instrumentation  solution  that  it  was  designed  to  be.  A  number  of  roadblocks  have  been 
overcome  along  the  way.  Although  the  initial  goal  of  non-invasiveness  has  not  been 
completely  met,  until  aircraft  are  designed  to  be  non-invasively  instrumented,  any  similar 
system  will  face  the  same  problems.  With  AIS,  the  evaluation  procedures  need  not  be 
redesigned  for  every  EC  system  test.  AIS  is  very  flexible,  as  illustrated  by  the  number  of 
aircraft  and  systems  on  which  it  has  been  and  will  be  used.  Finally,  with  the  development 
of  CAPS/MARS,  when  some  pre-test  work  has  been  done  specific  to  the  aircraft  and 
system  under  test,  rapid  turnaround  has  been  demonstrated.  When  AIS  is  used  to  support 
EC  T&E,  it  improves  the  quality  of  the  evaluation  and  permits  standardization  throughout 
the  test  process. 


^  MARS  3.2  does  not  have  the  capability  to  correlate  airborne  interceptor  actions  to  EC  system 
responses.  Some  of  this  capability  has  been  developed,  but  completion  of  this  upgrade  will  likely 
require  sponsorship  from  a  program  needing  this  analysis. 
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AAA 

Anti-Aircraft  Artillery 

AAM 

Air-to-air  Missile 

ABI 

Advanced  Bus  Interface 

ACETEF 

Air  Combat  Environment  Test  and  Evaluation  Facility 

ACMI 

Air  Combat  Maneuvering  Information 

ADAS 

Alternate  Data  Acquisition  System 

ADRS 

Automated  Data  Reduction  System 

AIS 

Airborne  Instrumentation  System 

ATDOP 

Advanced  Test  Data  Optimal  Processor 

BIT 

Built-in-Test 

C/A 

Coarse/ Acquisition 

CAPS 

Common  Airborne  Processing  System 

DBRS 

Digital  Bus  Recorder  System 

DF 

Direction  Finding 

DFR 

Digital  Flight  Recorder 

DLL 

Dynamic  Link  Library 

DOT&E 

Director,  Operational  Test  and  Evaluation 

EC 

Electronic  Combat 

ECM 

Electronic  Countermeasures 

EW 

Electronic  Warfare 

FCR 

Fire  Control  Radar 

FOG 

Fiber  Optic  Gyroscope 

GPS 

Global  Positioning  System 

GTRI 

Georgia  Tech  Research  Institute 

ICD 

Interface  Control  Document 

IMU 

Inertial  Measurement  Unit 

INS 

Inertial  Navigational  System 

MARS 

Mission  Analysis  and  Reporting  System 

NPU 

Navigational  Processor  Unit 

OPD 

Output  Product  Description 

OT&E 

Operational  Test  and  Evaluation 

PASS 

PCMCIA 

PRI 

Protocol  Analysis  Simulation  System 

Personal  Computer  Memory  Card  Industry  Association 
Pulse  Repetition  Interval 

RASDAS 

RF 

RWR 

Record  All  Small  Data  Acquisition  Systern 

Radio  Frequency 

Radar  Warning  Receiver 

SAM 

Surface-to-air  Missile 

T&E 

TEWS 

TIP 

TSPI 

Test  and  Evaluation 

Tactical  Electronic  Warfare  System 

Test  Instrumentation  Pod 

Time,  Space,  and  Position  Information 

UTC 

Universal  Coordinated  Time 

VTR 

Videotape  Recorder 
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